fix(plugin-broker): drain pending alerts and wait for in-flight deliveries on shutdown#4488
Conversation
|
@andreabravetti: There are no 'kind' label on this PR. You need a 'kind' label to generate the release automatically.
DetailsI am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository. |
|
@andreabravetti: There are no area labels on this PR. You can add as many areas as you see fit.
DetailsI am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository. |
|
/kind fix |
|
I'm having trouble choosing the right area, perhaps a mix of agent and cscli? |
…eries on shutdown When cscli notifications test kills the plugin tomb immediately after enqueuing an alert, the broker's select loop can pick the Dying() channel before reading PluginChannel, silently dropping the notification. The inner loop after watcher.Kill() also races Dead() vs PluginEvents, losing alerts queued by addProfileAlert. Fix with a 3-step shutdown sequence: 1. Drain PluginChannel 2. Flush alertsByPluginName map 3. Wait for in-flight delivery goroutines via sync.WaitGroup before Kill() Also wrap PluginEvents delivery in a goroutine with WaitGroup tracking.
andreabravetti
force-pushed
the
fix/plugin-broker-shutdown-race
branch
from
7084f31 to
8534376
Compare
|
I just fixed this PR, there was a mix of two works in the same branch, sorry about that. Now it should be clean. |
1 participant
When cscli notifications test kills the plugin tomb immediately after enqueuing an alert, the broker's select loop can pick the Dying() channel before reading PluginChannel, silently dropping the notification. The inner loop after watcher.Kill() also races Dead() vs PluginEvents, losing alerts queued by addProfileAlert.
Fix with a 3-step shutdown sequence:
Also wrap PluginEvents delivery in a goroutine with WaitGroup tracking.
Fixes issue #4487